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I.  INTRODUCTION 


A new  method  of  enhancing  the  detectability  of  narrowband  signals  in  the  presence 
of  broadband  noise  has  recently  been  introduced  [ 1,2,3] . This  method,  called  “adaptive 
line  enhancement,”  is  based  on  the  Widrow-Hoff  least-mean-square  (LMS)  adaptive  algorithm. 
As  shown  in  Fig.  1 , it  is  implemented  with  a two-channel  processor  in  which  a delayed  version 
of  the  input  data  is  adaptively  filtered  and  subtracted  from  the  instantaneous  input  data. 

The  difference  between  the  delayed  and  instantaneous  data  serves  as  the  error  signal  e(j)  for 
the  LMS  algorithm,  which  governs  the  frequency  response  of  the  adaptive  filter  in  such  a way 
that  the  power  of  eO)  is  minimized. 

Operation  of  the  adaptive  line  enhancer  (ALE)  of  Fig.  1 can  be  understood  intuitively 
as  follows.  The  delay  causes  decorrelation  between  the  noise  components  of  the  input  data 
in  the  two  processor  channels  while  introducing  a simple  phase  difference  between  the  sinus- 
oidal components.  The  adaptive  filter  responds  by  forming  a transfer  function  equivalent  to 
that  of  a narrowband  filter  centered  at  the  frequency  of  the  sinusoidal  components.  The 
noise  component  of  the  delayed  input  is  rejected,  while  the  phase  difference  of  the  sinusoidal 
component  is  readjusted  so  that  they  cancel  each  other  at  the  summing  junction,  producing 
a minimum  error  signal  composed  of  the  noise  component  of  the  instantaneous  input  data 
alone. 

Use  of  the  ALE  to  detect  sinusoidal  signals  in  uncorrelated  or  “white”  noise  is  dis- 
cussed in  [ 1 1 , [2] , and  [3]  .*  In  this  application  any  value  A of  delay  can  be  chosen.  The 
ALE  can  also  be  used  to  detect  sinusoidal  signals  in  correlated  or  “colored”  noise  [2] . In 
this  case  it  is  often  necessary  to  choose  a relatively  large  value  of  A to  insure  decorrelation 
between  the  noise  components  in  the  two  processor  channels. 

The  ALE  with  unit  delay  (A  = 1 ) is  identical  to  the  instantaneous  frequency  tracker 

of  [4]  The  ALE  and  instantaneous  frequency  tracker  are  further  shown  in  [5]  to  be  forms 
of  adaptive  linear  prediction  filter.  A comprehensive  treatment  of  linear  prediction  filtering 
techniques  is  given  in  [6]  and  [7] . The  LMS  algorithm  was  first  applied  to  the  design  of  an 
adaptive  prediction  filter  in  [ 14] . A recent  hardware  implementation  of  an  8-weight  LMS 
adaptive  linear  predictor  and  its  application  to  instantaneous  frequency  estimation,  speech 
encoding,  and  other  similar  problems  are  discussed  in  [8] . 

The  advantages  of  adaptive  line  enhancing  with  respect  to  conventional  digital 
Fourier  analysis  have  been  discussed  with  some  controversy  in  the  literature.  A comparison 
in  [ 1 ] , using  figures  of  merit  based  on  output  signal-to-noise  ratio,  indicated  that  the  ALE 
required  less  data  to  achieve  a given  level  of  detection  performance.  This  comparison  was 
shown  to  be  in  error  in  [9] , and  a correction  presented  in  [10]  indicated  that  performance 
with  a given  amount  of  data  was  equivalent.  A definitive  comparison  based  on  statistical 
detection  theory',  however,  has  not  yet  been  published.  Considering  the  fundamental 


^Detection  is  accomplished  by  digital  Fourier  analysis  of  the  adaptive  filter  weights  [1 ,2]  or  adaptive  filter 
output  [3] . 

^In  this  application  a modified  maximum  entropy  spectral  estimate  is  computed  from  the  adaptive  filter 
weights. 
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Figure  1 . Adaptive  line  enhancer  (ALE). 
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differences  between  the  two  methods  it  appears  likely  that  each  will  have  advantages  in 

different  applications.^  As  suggested  in  ( 1 1 J , the  ALE  in  many  applications  is  effective 
over  a wide  range  of  input  signal  and  noise  parameters  with  tittle  a priori  information.  Both 
(10)  and  [111  have  also  emphasized  its  ability  to  resolve  closely  spaced  narrowband  signals. 

Since  the  ALE  is  a new  type  of  processor,  many  questions  regarding  its  performance 
and  advantages  remain  to  be  answered.  The  goal  of  this  paper  is  to  analyze  its  steady-state 
behavior  with  a stationary  input  consisting  of  multiple  sinusoids  in  uncorrelated  noise.  It 
will  be  shown,  using  the  method  of  undetermined  coefficients,  that  the  L X L Wiener-Hopf 
matrix  equation  describing  the  steady-state  impulse  response  of  an  L-weight  ALE  with 
arbitrary  delay  or  “prediction  distance”  A may  be  transformed  into  a set  of  2N  coupled 
linear  equations,  where  N is  the  number  of  sinusoids  in  the  ALE  input.  This  set  of  equations, 
which  decouples  as  the  adaptive  filter  becomes  longer,  provides  a useful  description  of  the 
interaction  between  the  sinusoids  introduced  by  the  finite  length  of  the  filter.  The  derived 
impulse  responses  and  transfer  functions  form  an  analytical  basis  for  comparing  in  terms  of 
L and  A the  various  techniques  of  obtaining  spectral  estimates  with  the  ALE  for  this  class  of 
input  signals.  These  results  also  offer  a means  of  comparing  spectral  estimates  obtained  with 
adaptive  line  enhancing  and  those  obtained  with  other  techniques. 


II.  ANALYSIS 

A.  Determination  of  Appropriate  Models 

The  Widrow-Hopf  LMS  algorithm  for  the  ALE  is  as  follows  [ 14] : 

Wj+](k)  = Wj{k)  + 2n  j^x(j)  xG  - A - k)  - xO  - A - k) 

• ^ xO  - A-e)wj(C)1,k  = 0,  1,  ...  ,L  - 1 (I) 

where  Wj(k)  is  the  update  of  the  kth^  weight  of  the  ALE;/i  is  a scalar  representing  the 

influence  of  the  input  x(j)  on  the  0 + I update  of  w(k);  and  L and  A are,  respectively,  the 
number  of  weights  and  the  delay. 

For  uncorrelated  stationary  inputs  it  has  been  shown  [4, 14, 15]  that,  starting  with 
an  arbitrary  initial  weight  vector,  the  expected  value  Wj(k)  in  (1 ) converges  to  the  solution  of 

the  Wiener-Hopf  matrix  equation,  provided  that  0 <(i  < 1 where  Xniax  largest 

eigenvalue  of  the  data  autocorrelation  matrix: 

lim  ElWj(k)]  = w*(k),  k = 0,  1 , . . . , L - 1 (2) 

j-^ 

^Comparisons  of  various  spectral  estimation  methods,  including  Fourier  and  maximum  entropy  methods  (but 
not  adaptive  line  enhancing),  are  presented  in  [12]  and  [13] . Performance  is  shown  to  depend  on  a number 
of  factors,  including  the  amount  of  data  available  and  the  input  signal-to-noise  ratio. 
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where  E ( -1  denotes  expectation  and  w*(k)  is  the  component  of  the  L-dimensional 
vector  W*,  which  is  the  solution  of  the  Wiener-Hopf  matrix  equation: 


(3) 


R-W*  = P. 

In  (3)  R is  the  L X L data  autocorrelation  matrix  with  elements  (R)g|j  = ^xx^®  ~ 

= E (x(j)  x(j  + C)I  is  the  autocorrelation  function  for  x(j),  and  P is  the  L-dimensional 
vector  with  components  (P)j^  = 

Further,  for  the  case  of  uncorrelated  input  vectors,  the  covariance  of  the  weight- 
vector  noise  in  steady  state  has  been  shown  ( 1 , App.  D|  to  be  given  by 

lim  covlWj-W*]=p{^i„i  (4) 

j— K)0 


where 


L-1 

Un  = </'xxW-l  + 

k=0 


(5) 


is  the  minimum  mean  square  error  of  the  finite  length  Wiener  filter  W*  and  J^is  the  L X L 
identity  matrix.  The  weight  vector  noise  is  inherent  in  the  estimation  process  used  in  the 
LMS  algorithm  and  results  from  performing  spectral  estimates  in  a finite  observation  time. 
(Note  that  an  exact  solution  of  the  Wiener-Hopf  matrix  equation  requires  an  infinite  time 
history  of  data,  since  the  exact  values  of  required.)  Tlie  weight  vector  noise 

results  in  an  excess  steady-state  mean-square-error  output,  so  that  the  ALE  error  never  reaches 
the  theoretical  minimum 

A useful  measure  of  the  difference  between  actual  and  optimal  performance  is  the 
“misadjustment,”  defined  as  the  dimensionless  ratio  of  the  average  excess  mean  square  error 
to  the  minimum  mean  square  error  1 16] : 

_ average  excess  mean  square  error 
^min 

The  theoretical  expression  for  M derived  in  [16]  is 


M = ^L«xxW- 

Note  that  (7)  was  derived  under  the  assumption  that  the  input  data  are  uncorrelated  and  tends 
to  be  smaller  than  the  measured  misadjustment  when  the  data  are  correlated.  Note  also  that 
the  magnitude  of  the  excess  error  can  be  made  arbitrarily  small  by  making  //  arbitrarily  small, 
since  a decrease  in  n results  in  an  increase  in  integration  time  during  the  adaptive  process. 

The  weight  vector  noise  produces  noise  in  the  ALE  transfer  function.  Assuming 
Gaussian  statistics  for  the  weight  vector  noise,  it  has  been  shown  ( 1 , App.  D]  that  when  the 
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input  noise  power  in  xQ)  is  large  compared  to  the  input  signal  power,  the  noise  in  the 
transfer  function  has  zero  mean  and  a steady-state  power  which  is  given  by 


t 

I 

I, 


Lfiv- 


(8) 


'y 

where  v‘-  is  the  input  noise  power.  It  has  also  been  shown  [14,  16]  by  diagonalizing  R that 
transients  in  the  mean  weight  vector  consist  of  sums  of  exponentials  with  time  constants 
given  by 


T ^ 

P 2n\ 


(9) 


where  Xp  is  the  p^  eigenvalue  of  R.  In  general,  a precise  treatment  of  the  dynamics  of  the 

LMS  algorithm  is  complicated  due  to  the  difficulties  in  diagonalizing  R.  Note,  however,  that 
the  dynamics  of  E[Wj(k)l  for  the  important  case  of  a sinusoid  in  white  noise  have  been 

described  in  [17]  by  diagonalizing  R. 

Among  the  first  to  consider  the  convergence  of  (1 ) for  correlated  data  was  Senne 
[18] , who  demonstrated  that  E[Wj(k)]  does  not  necessarily  approach  w*(k)  when  xG)  is 

correlated.  Further,  Senne  demonstrated  that  both  the  asymptotic  excess  mean  square  error 
and  the  time  constants  Tp  increase  as  correlation  increases.  Daniell  [19]  showed  that, 

although  the  weight  vector  may  be  biased  away  from  W*  in  steady  state,  it  can  be  made 
arbitrarily  close  to  W*  in  the  sense  that  given  any  number /3>0  there  exists  a p^>0  such 

that  for  all  0 < /i  < 

lim  sup  E[|Wi  -W*|2]  </J. 

J-^OO 

The  assumptions  made  by  Daniell  included  ergodicity  as  well  as  asymptotic  independence 
and  uniform  boundedness  of  the  conditional  moments  of  the  observable  given  the  past  data. 
As  noted  by  Kim  and  Davisson  [20] , this  last  assumption  is  rather  strong  and  is  not  satisfied 
in  a number  of  important  cases.  In  [20]  Kim  and  Davisson  have  also  considered  the  con- 
vergence properties  of  adaptive  algorithms  similar  to  (IX  for  correlated  stationary  input  data 
and  were  able  to  show  that,  under  certain  assumptions  concerning  the  input  data,  the  mean 
norm-square  of  the  weight  vector  deviation  converged  to  asymptotic  bounds  which  can  be 
made  arbitrarily  small  by  decreasing  the  adaptation  constant.  However,  the  derivations  in 
[20]  require  that  the  adaptive  algorithm  form  its  gradient  estimate  by  some  averaging  of  the 
input  data  as  opposed  to  using  instantaneous  data  values  as  in  (1 ).  Therefore,  the  results  in 
[20]  do  not  strictly  apply  to  (1).  Other  results  and  references  on  the  convergence  proper- 
ties of  (1 ) for  stationary  correlated  input  data  are  discussed  in  [17]  and  [29] . 

Even  though  general  convergence  properties  of  ( 1 ) for  stationary  correlated  input 
data  are  very  difficult  to  obtain,  it  is  still  reasonable  to  assume  that,  for  suitably  small  values 
of  ii,  ElWj(k)]  will  converge,  within  a good  approximation,  to  w*(k)  in  many  cases  of 

practical  interest.  This  assumption,  as  noted  in  [ 1 ] , generally  holds  in  practice,  and  it  will 


be  used  in  this  paper  to  derive  the  expected  values  of  thv.  ;;eady-state  ALE  weights  for 
stationary  inputs  consisting  of  multiple  sinusoids  in  white  noise.  To  verify  that  the 
analytical  results  obtained  from  the  finite  length  Wiener  filter  model  (3)  do  in  fact  describe 
the  steady-state  performance  of  the  ALE,  experimental  results  have  been  obtained  using  a 
hardware  implementation  of  the  ALE  for  several  special  cases.  The  analytical  and  experi- 
mental results  are  compared  in  Section  III  and  are  shown  to  be  in  good  agreement. 


B.  Steady-State  Impulse  Response  and  Transfer  Function  of  the  ALE  for  Multiple  Sinusoids 
in  White  Noise 

Using  the  Wiener-Hopf  model  for  the  ALE  response,  (3)  may  be  expressed  in  compo- 
nent form  as 


L-1 

^ = + A).0<C<L- 1.  (10) 

k=0 


When  x(j)  consists  of  N sinusoids  in  white  noise, 

N 

^xx^^^  ^ ®n^ ^n*^ 

n=l 

where  6(k)  is  the  Kronecker  delta  function  (j^.,  6(k)  = 0 if  k + 0 and  6(0)  = 1 ).  In  (1 1) 
is  the  power  in  the  nth  sinusoid,  represents  the  frequencies  of  the  sinusoids,  and  an“  is 
the  white  noise  power.  If  oq^4  0.  then  R is  positive  definite  (as  shown  in  the  Appendix), 
and  a unique  solution  to  (3)  exists. 

Tire  inverse  of  R can  be  calculated  explicitly,  for  the  special  case  when  0xx^*^^ 

given  by  ( 1 1 ),  by  the  repeated  use  of  a well-known  matrix  inversion  identity  sometimes 
referred  to  as  Woodbury’s  identity  (21 1 . This  identity  has  been  used  by  a number  of  authors 

to  analyze  R“*  and/or  W*  when  is  in  the  form  of  (1 1 ).  Zeidler  and  Chabries  [3] 

used  this  identity  to  analyze  the  steady-state  properties  of  the  ALE  for  one  sinusoid  in  white 
noise,  and  the  use  of  the  identity  has  been  discussed  by  Lacoss  (12],  Frost  [ 13 1 , and  Marple 
(301  for  purposes  of  examining  different  methods  of  spectral  estimation.  This  inversion 
identity  has  also  been  used  by  Shapard  et  al.  [22]  for  purposes  of  adaptive  matrix  inversion; 
by  Edelblute  et  al.  [23]  to  analyze  different  criteria  for  optimization  of  acoustic  signal 
detection;  and  by  Capon  [24]  to  analyze  a technique  for  high  resolution  frequency- 
wavenumber  spectrum  analysis.  The  application  of  Woodbury’s  identity  to  obtain  W*  is, 
however,  quite  tedious  and  does  not  resolve  many  questions  concerning  the  analytic  struc- 
ture of  w*(k)  if  N is  much  larger  than  two. 

In  this  paper  it  will  be  shown  that  solutions  for  w*(k)  which  provide  insight  into  the 
basic  analytic  structure  of  the  steady-state  ALE  impulse  response  for  an  arbitrary  number  of 
sinusoids  can  be  obtained  by  an  alternate  technique  called  the  method  of  undetermined 
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coefficients.  The  method  of  undetermined  coefficients  assumes  a solution  for  w*(k)  in 
terms  of  unknown  constants  and  substitutes  this  assumed  solution  into  (10)  to  obtain  a set 
of  equations  for  the  unknown  constants  [25,  26  J.  Since  a unique  solution  to  (10)  exists  as 

long  as  oq“+  0,  a solution  to  the  set  of  equations  for  the  unknown  constants  provides  the 

unique  solution  to  (10).  It  will  be  shown  that  for  an  L-weight  filter  with  N sinusoids  present 
in  the  input  data  sequence,  the  method  of  undetermined  coefficients  effectively  transforms 
the  set  of  L coupled  equations  in  ( 10)  into  a set  of  2N  coupled  equations.  This  then  leads  to 
a simpler  set  of  equations  whenever  L>  2N.  (A  procedure  similar  to  this  was  introduced  by 
Zadeh  and  Ragazzini  [25]  for  solving  the  integral  equations  (analogous  to  (10))  for  the 
optimum,  finite  length,  continuous  predictive  filter.  This  analytical  technique  was  also  used 
by  Morgan  and  Craig  [8]  to  solve  (10)  for  one  sinusoid  in  white  noise.) 

Tlie  form  ot  the  assumed  solution  for  w*(k)  for  N sinusoidal  inputs  of  the  form 
specified  in  ( 1 1 ) is 


2N 

n=l 


(12) 


where  for  notational  convenience  is  defined  as  -to„(n  = 1 . 2, . . . , N);  the  cOp+jvj  are 

thus  the  negative  frequency  components  of  the  input  sinusoids.  Substituting  (1 2)  into  (10) 
with  given  by  ( 1 1 ),  and  equating  coefficients  of  exp  (jco^C)  (for  r = 1,2,...,  2N)  in 

the  resulting  equation,  leads  to  the  following  2N  equations  in  the  2N  constants  A| , 

^2 ^2N- 


r ^ "yrn  "^n  “ ~T  YT^' ^ 


n=l 

nfr 


2N 


where  in  ( 1 3)  is  defined  as  (n  = 1 , 2,  . . .,  N)  and  7—  is  given  by 


T'rn 


L + 2ao2/ff^2  j_Ka;„-cOr) 


(13) 


(14) 


The  solution  of  (13)  for  the  A„  completely  determines  w*(k)  through  (12).“^  Since  N is 

much  smaller  than  L in  many  cases  of  practical  interest,  (13)  often  provides  a much  simpler 
set  of  equations  than  (10). 

A number  of  interesting  analytic  properties  of  w*(k)  can  be  observed  through 
(12)-(14).  First,  (12)  implies  that  when  the  input  to  the  ALE  consists  of  N sinusoids  and 
additive  white  noise,  the  mean  steady-state  impulse  response  of  the  ALE  can  be  expressed 
as  a weighted  sum  of  the  positive  and  negative  frequency  components  of  the  input  sinusoids. 

^As  shown  in  the  Appendix,  ( 1 3)  has  a unique  solution. 
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From  (14),  it  is  seen  that  the  coefficients  7^,  which  couple  the  A^,  together  in  (13),  are 

proportional  to  ( 1 - e )/( 1 - e " '^  ),  which  is  the  L-point  Fourier  transform 

of  exp  (jWjjk)  evaluated  at  cOp  Note  that  from  the  form  of  the  7j.j^  it  follows  that 

An+N  ~ (n  = 1,  2, . . . , N),  where  the  overbar  denotes  complex  conjugate.  This  relation 
is  of  course  necessary  to  insure  that  w*(k)  is  real.  A plot  of  l7^„|  versus  Acjf, ^ = cOj,  - ojp 

given  in  Fig.  2(a)  for  two  different  filter  lengths  and  for  2oq“/o^^  = 1 , shows  that,  when  L 
is  large  or  when  is  some  integral  multiple  of  27r/L,  7^^  can  be  neglected.  When 

'^^nr  Trn  (L/2)(o^“/oq")/(  1 + (L/2)(aj.“/aQ“));  and  further,  as  L becomes  large,  the 
ratio  of  the  peaks  of  l7rnl  at  Acj^r  = (2p  + 1 ) rr/L  to  (7^1  at  Aoj^^  = 0 for  p = 1 , 2, . . . is 

given  approximately  by  l/(p  + 1/2)^.  The  first  two  ratios,  for  instance,  are  0.212  and 
0. 1 27.  Therefore,  even  if  a particular  Acj^j.  is  close  to  the  filter  resolution,  i.e.,  within  the 

first  several  peaks  of  Fig.  2(a),  the  associated  7^^  could  possibly  be  neglected  in  (13)  without 
creating  serious  error  in  w*(k),  especially  if  (L/2)(o^“/oq“)/(1  + (L/2)(ar^/jQ^))  is  much 
smaller  than  1 . A specific  case  when  some  of  the  7^^  can  be  neglected  is  presented  in 
Section  11. C.  Note  also  that  as  aj.“/aQ“  becomes  very  small  the  7^p  approach  zero.  This  is 
indicated  in  Fig.  2(b),  where  l7^pl  is  plotted  for  two  different  filter  lengths  and  for 
200^/0^2  = 50. 

As  7pp  0 for  all  n and  r (i.e.,  as  L becomes  large)  the  Ap  uncouple  and  are  given  to 
a good  approximation  by 


L+2<7q2/„^^ 


Eq.  (1 5)  is  identical  to  the  expression  for  the  amplitude  of  the  mean  steady-state  ALE  im- 
pulse response  for  one  sinusoid  at  cOp  in  white  noise  previously  derived  in  [ 1 ] and  [3] . 

Therefore,  as  the  7^p  -*•  0,  the  ALE  for  N sinusoids  will  adapt  to  a linear  superposition  of  N 

independent  ALE’s,  each  adapted  to  a single  sinusoid  in  white  noise.  From  (12),  it  is  seen 
that  the  frequency  response  of  the  steady-state  ALE,  which  will  be  denoted  by  H*(u>),  can 
be  simply  expressed  in  terms  of  the  Ap: 

L-1 

H*(cj)  = J w*(k)  ' ) 

k=0 


KWp-w)L 

j(Wn-a;) 
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J 


n f 


Figure  2.  Relation  of  coupling  coefficients  7„  to  difference  frequencies  of  simuaoidal  inputs 

(a)2ao2/a,2=l.  (b)2ao2/o^2,50 


II 


As  L becomes  large,  so  that  (1 5)  is  valid,  H*(co)  is  given  to  a good  approximation  by 


H*(co)  ^ y — 

n=lL  + 


N -j(cOn  A + w) 


2 OQ^Ia^ 


N j(w  A - cj) 
n=lL  + 


2 OQ^/On^ 


-j(co_  + co)L 
1 - e 

-j(Wn  ^ 

1 - e " 

j(o;n  - co)L 
1 -e  ” 

j(cOn  -co) 

1 - e " 


(17) 


Eq.  (17)  corresponds  to  a sum  of  bandpass  filters  (centered  at  ±00^),  each  having  a peak  value 
given  by  (L/2)  SNRj^/((L/2)  SNR^  + 1 ),  where  SNR^^  = a^ja^.  As  L °o,  all  of  the  peak 

values  in  (17)  approach  1,  and  the  ALE  becomes  a linear  superposition  of  perfectly  resolved 
bandpass  filters,  each  with  unit  gain  at  its  center  frequency.  Caution  must  be  exercised  in 
choosing  L,  however,  because  as  L is  increi;sed,  the  weight  vector  noise  is  also  increased,  as 
indicated  by  (8),  and  the  performance  of  the  ALE  can  be  degraded.  Therefore,  in  practice, 
a value  for  L which  provides  a trade-off  betv^een  weight  vector  noise  and  enhancement 
capabilities  should  be  chosen. 


C.  Approximate  Expressions  for  the  Wiener  Filter  Solution  when  Specific  Interaction  Terms 
Are  Negligible 

As  shown  in  Section  II. B,  the  exact  Wiener  weight  vector  solution  for  N sinusoidal 
inputs  in  white  noise  can  be  obtained  by  solving  a set  of  2N  coupled  linear  equations  (13). 
For  cases  in  which  specific  interaction  terms  are  known  to  be  negligible  it  may  be  possible 
to  further  reduce  (13)  to  a smaller  set  of  coupled  equations  which  only  involve  the  non- 
negligible  and  provide  a valid  approximation  to  the  exact  Wiener  weight  vector.  For 

example,  consider  the  case  of  two  sinusoids  in  white  noise  in  which  there  is  little  interaction 
between  the  positive  and  negative  frequency  components  of  the  sinusoids  but  appreciable 
interaction  between  the  two  positive  frequencies  (and  therefore  between  the  two  negative 
frequencies).  For  this  case,  the  correlation  function  is  given  by  ( 1 1 ) with  N = 2. 

0xx^®^  = 6(C)  + Oj^  cos  cojC  + a-^  cos  (*8) 

Therefore,  from  (12-14),  w*(k)  is  given  by 

W(k)  = A,  A2  » Aj  + A4  r-'k  „„ 

where  0^3  = -cu  j ; <04  = -002;  and  the  Aj^  are  linearly  coupled  together  through  the  1 2 
coupling  coefficients  (r,  n = 1 , 2, . . . , 4;  r+  n),  which  are  given  by  (14).  Since  the 
interaction  between  the  positive  and  negative  frequency  components  is  assumed  to  be  small. 
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the  8 coefficients  7J3,  731, 714,  741. 723>  724>  732’  "^42  neglected,  and  (13) 

reduces  to  the  two  independent  sets  of  2 X 2 linear  equations: 


jtjjA 

0 * 


T'rn  ^n 


n=l 

nfr 


L + Ioq^Io/ 


, r=  1,2 


(20) 


n=3 

nfr 


(21) 


The  solution  of  (20)  and  (21)  gives  the  following  approximations  for  the  A„: 


A,  = AoSe 


1 


1 - ^3  = 1 


At  - A^  25 


'yi2'l'21 


2 - ^4  = iT 


'y\2'Y2l  1l+2oo^/o2^  L+2oo^/oi 


2,„  2 


(22) 


where  7|2  and  are  given  by  (14).  As  a numerical  example  illustrating  the  difference 
between  the  approximate  solution  of  (19)  - (22)  and  the  exact  solution,  consider  the  case 
when  oj|/27r  = 0.25;  cu2/2ir  = 0.26;L  = 16;A  = 5;  and  SNRj  (=  a | ~ SNR2 

(=  oj^Ioq^)  = 1 .0.  For  this  case,  the  magnitudes  of  the  coupling  coefficients  between  the 
positive  and  negative  frequencies  are  I724I  = I742I  " 0.047;  I723I  = I732I  ~ ~ 1741 1 “ 

0.027;  I713I  = I731 1 = 0.  The  magnitude  of  the  coupling  coefficients  between  the  closely  | 

spaced  lines  is  l7i  2I  “ 1721 1 “ 1734^  ” I743I  “ 0.86.  In  Fig.  3 plots  of  w*(k)  derived  from  j 

( 1 2)  - ( 1 4)  and  from  the  approximations  ( 1 9)  and  (22)  are  presented.  As  is  seen,  the  greatest  | 

discrepancy  is  approximately  10  percent  and  occurs  at  the  peaks  where  the  four  sinusoids  in  ' 

( 1 9)  tend  to  add  constructively.  j 


III.  EXPERIMENTAL  RESULTS 

In  this  section  we  apply  the  analytical  results  obtained  in  Section  II  for  N sinusoids 
in  uncorrelated  noise  to  the  specific  cases  of  a single  sinusoid  in  white  noise  and  two 
sinusoids  in  white  noise.  Experimental  verification  that  the  steady-state  response  of  the  ALE 
is  in  fact  described  by  the  Wiener  filter  model  for  these  cases  is  provided.  The  experimental 
date  presented  in  this  paper  were  obtained  on  a variable  length  hardware  implementation  of 
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Figure  3.  Comparison  of  weight  values  derived  from(12)-(14)and  from 
approximations  (19)  and  (22)  for  a particular  case  defined  in  the  text. 

the  ALE  which  employed  9-bit  linear  quantization  in  the  upper  (input)  channel  and  4-bit 
log  quantization  in  the  adaptive  filter  (reference)  channel  of  Fig.  1.  Filter  lengths  of  8,  16, 
32,  64,  1 28,  and  256  weights  can  be  obtained  with  this  hardware.  Experimental  plots  of  the 
frequency  response  of  the  ALE  in  steady  state  were  obtained  by  “freezing”  the  weights  at  a 
particular  instant  (after  adaptation),  and  applying  white  noise  to  the  input  of  the  stationary 
filter  and  spectral-analyzing  the  resulting  output.  The  value  of  5;peri- 

ments  was  sufficiently  small  that  weight  vector  noise  was  negligible.  The  noise  spectrum  was 
obtained  from  a thermal  noise  generator  whose  output  was  filtered  by  a -48-dB  per  octave 
low-pass  filter  with  its  -3-dB  point  at  1.8  kHz.  The  input  noise  spectrum  was  thus  only 
approximately  flat  in  the  region  of  interest.  The  major  sources  of  experimental  error  were 
the  low-pass  characteristics  of  the  input  noise  spectrum  and  quantization  noise.  Input  levels 
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for  all  experiments  were  controlled  to  insure  that  the  error  due  to  quantization  noise  was 
approximately  2 percent.  All  the  experimental  curves  presented  in  this  section  were  obtained 
directly  from  x-y  pen  plots  of  the  spectrum  analyzer  output.  All  the  theoretical  curves  were 
obtained  by  evaluating  the  quantity  of  interest  at  10  X L different  frequency  values  between 
tJ  = 0 and  cj  = jr.  The  input  sampling  rate  was  3.9  kHz  in  all  cases. 


A.  One  Sinusoid  in  White  Noise 

For  this  case  the  correlation  function  given  by 


0xxW  = 6(C)  + cos  OJjC. 

Therefore,  from  (12)-(14),  w*(k)  is  given  by 

iojik  -j^'^ik 

w*(k)  = Aie'  * +A2e  ' 


(23) 


(24) 


where 


Al 


1 


1 


jcoiA  -jwiA 

[e  -Tl2® 


(25) 


and 


1 


712  = 


L + 2aQ‘‘/oj‘ 


-2jcujL 
1 -e  * 

-2Ja;, 

1 -e 


(26) 


In  this  particular  case,  7|  2 represents  coupling  between  the  positive  and  negative  frequency 
components  of  the  real  sinusoid.  Note  that  as  7j2  0,]^.,  as  the  frequency  components 

become  uncoupled,  w*(k)  is  given  to  a good  approximation  by 

SNR 

-TTsSroTT) 

with  SNR  = a^/aQ.  Therefore,  as  long  as  there  is  negligible  coupling  between  the  positive 

and  negative  frequency  components  of  the  sinusoid,  the  steady-state  impulse  response  of 
the  ALE  is  a sampled  cosinusoid  at  frequency  W|  which  has  been  shifted  in  phase  by  cui  A. 

The  amplitude  of  the  cosinusoid  is  SNR/(1  + SNR(L/2)).  This  result  was  also  obtained  in 
(3)  and  discussed  in  ( 1 , App.  D) . However,  as  7j2  becomes  appreciable,  the  amplitude  and 

phase  shift  of  the  steady-state  ALE  impulse  response  begin  to  change  considerably,  as  is 
observed  in  (24)  - (26). 


Fig.  4 represents  the  results  of  an  experiment  carried  out  on  a 1 6-weight  ALE  with 
A = 8 samples,  = 5.12  X 10~^,  and  an  input  sinusoid  with  SNR  = oj^/oq^  = 0.557 

and  frequency  (jiYlIv  = 0.0039.^  For  this  frequency  there  is  appreciable  coupling  between 

the  positive  and  negative  frequency  components  of  the  sinusoid.  Experimental  plots  of  the 
input  and  ALE  output  power  spectrum  (plotted  on  the  same  relative  log  scale  and  denoted 
by  Sx(oj)  and  Sy(to),  respectively)  are  presented  in  Fig.  4(a).  In  Fig.  4(b)  three  curves  are 

presented.  The  first  two  curves  are  experimental  and  theoretical  linear  plots  of  |H*(co)|  and 
the  third  is  a linear  plot  of  the  function 


lG(a))i  = - 

L + 2oQ^toi^ 


-jcoj  A 
e 


-j(toi  + a;)L 
1 -e  ' 

j _^-j(w,  +CO) 


J ^jfcoj  - a))L 

1 _^KC0J  - CO) 

where  A = 8.  The  term  |G(oj)|  in  (28)  is  the  Fourier  transform  of  w^fk)  with  71-1  = 0.  This 
curve  is  included  to  show  the  effects  of  the  coupling  coefficient  7p  on  the  frequency 

response  of  the  steady-state  ALE.  As  is  seen,  the  effect  of  the  coupling  coefficient  is  to 
reduce  the  peak  values  and  null  depths  in  |H*(cj)1.  Neglecting  the  coefficient  when 
u>|  « 27r/L  can  give  rise  to  serious  errors,  as  is  observed  in  Fig.  4(b).  It  should  be  noted 

that  there  is  excellent  agreement  between  the  theoretical  and  experimental  curves  of 
|H*(<aj)|  with  respect  to  the  location  of  the  peaks  and  nulls.  There  is  some  discrepancy  in 
the  two  curves,  however,  with  respect  to  the  amplitudes  of  |II*(co)|,  especially  in  the  higher 
frequency  ranges.  Tliis  discrepancy  is  mainly  due  to  the  nonflatness  of  the  noise  spectrum 
used  to  generate  the  experimental  curve.  As  a further  means  of  comparing  the  experimental 
ALE  steady-state  weight  vector  and  the  Wiener  weight  vector,  the  misadjustment  M defined 
by  (6)  was  determined  with  measured  values  of  the  steady-state  ALE  mean  square  error 
output.  For  the  case  represented  by  Fig.  4,  M was  found  to  be  approximately  1 percent. 

The  theoretical  misadjustment,  calculated  with  (7),  which  includes  gradient  estimation  noise 
only,  was  approximately  0.01  percent  for  this  case.  (In  these  experiments  gradient  estima- 
tion noise  was  a negligible  source  of  error  compared  to  quantization  noise  and  other  factors.) 

In  Fig.  5 experimental  and  theoretical  plots  of  20  log  |H*  (coj)!  are  presented  for 

different  input  SNR.  These  data  were  obtained  from  an  8-weight  ALE  with  A = 199 
samples  and  tjj  /2Tr  = 0.098.  Note  that  the  two  curves  agree  within  the  experimental  error 

of  the  measurements.  (The  bars  in  Fig.  5 denote  the  estimated  variances  of  the  measure- 
ments.) 


^In  this  section  all  frequencies  have  been  normalized  to  the  sample  frequency;  arrows  in  the  figures  denote 
the  actual  location  of  the  input  sinusoids. 
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(b) 


Figure  4.  Performance  of  16-weight  ALE  with  input  of  single  sinusoid  in  uncorre- 
lated noise;  delay  A=  8 samples;  = 5.12  x 10"^;  input  signal-to-noise  ratio 

SNR  = 0.557.  (a)  Power  spectra  of  input  Sjj(cj)  and  steady-state  output  Sy(to). 

(b)  Experimental  and  theoretical  steady-state  transfer  function  |H  (co)l  and  |Ci(cj)| 
defined  in  text. 
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B.  Two  Sinusoids  in  White  Noise 

For  the  case  of  two  sinusoids  in  white  noise  the  autocorrelation  function  of  the  input 
is  as  given  by  (18).  The  expected  value  of  the  steady-state  ALE  impulse  response  can  be 
expressed  as  a sum  of  sinusoids  as  in  (19),  where  the  amplitudes  of  the  sinusoids  Aj^  are  the 
solution  of  (13). 

Figs.  6-8  represent  the  results  of  three  experiments  carried  out  on  a 32-weight  ALE 
which  illustrate  the  effect  of  the  variable  prediction  distance  A on  the  impulse  response  and 
transfer  function  of  a finite  length  linear  predictive  filter.  In  these  experiments  the  frequen- 
cies of  the  sinusoids  were  = 0.0768  and  cu2/2rr  = 0.09984.  In  Figs.  6 and  7, 

~ = 1 ; in  Fig.  8,  aj“/oQ“  = I and  o-?Ioq  = 0.36.  Experimental  plots  of  the 

ALE  input  and  output  power  spectrum  (in  dB)  for  these  cases  are  presented  in  Figs.  6(a), 
7(a),  and  8(a).  Figs.  6(b)  and  8(b)  compare  the  experimentally  measured  transfer  function 
of  the  ALE  with  the  analytical  results  obtained  for  A = 6 samples,  L = 32,  and  the  frequen- 
cies and  SNR’s  of  the  two  sinusoids  defined  above.  Figs.  7(b)  and  7(c)  compare  the  experi- 
mentally measured  impulse  response  and  transfer  function  of  the  ALE  with  theory  for  the 
case  where  A = 1 9 samples  and  all  other  parameters  are  identical  to  those  chosen  in  Fig.  6. 

As  shown  in  Figs.  6(b)  and  8(b),  when  A = 6 samples  there  is  a deep  null  in  lH*(a))  I at 
oj  (u)j  + — CO jjy ; whereas,  when  A = 19  samples  (Fig.  7(c)),  the  null  is  practically 

gone.  The  null  at  A = 6 samples  and  the  two  highest  peaks  in  the  transfer  function  in 
Figs.  6(b)  and  8(b)  occur  in  spite  of  the  fact  that  the  frequency  resolution  of  the  filter  is  too 
coarse  to  resolve  the  two  input  sinusoids.  It  should  be  noted  that  the  phase  difference  be- 
tween the  sinusoidal  components  of  w*(k)  in  (19)  has  shifted  the  locations  of  the  two  high- 
est peaks  in  lH*(<o)  I away  from  the  frequencies  of  the  input  sinusoids,  as  indicated  in 
Figs.  6(b)  and  8(b). 

For  the  case  of  two  sinusoids  which  are  separated  in  frequency  by  Acu  = tj->  - cuj , 

the  delay  which  is  required  to  produce  the  null  effect  shown  in  Figs.  6 and  8 will  become 
larger  as  Aco  -*■  0.  To  verify  this,  note  that  lH*(a)gy)  I for  the  two  sinusoids  in  Figs.  6-8 

is  given  to  a good  approximation  by 


|H*(cUay)|  ss 


\.Q-iAuiLj2  ^ 1 _ejAcoL/2 

1 _ e-j-^w/2  , _gjAco/2 


(29) 


In  (29)  we  have  neglected  the  effects  of  the  negative  frequency  components  of  the  sinusoids 
(which  is  a good  approximation  for  the  parameters  in  Figs.  6-8).  Under  this  assumption, 

A|  and  A2  are  given  by  (22).  Eq.  (29)  can  then  be  expressed  in  the  following  form: 

lH*(cOgy)|  = F\/a^  + b^  + 2ab  cos  Acu((L  - 1 )/2  + A)  (30) 
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Figure  6.  Performance  of  32-weight  ALE  with  input  of  two  sinusoids  of  equal  power 
in  uncorretated  noise ; delay  A = 6 samples;  = 3.6  x 1 0”^.  (a)  Power  spectra 

of  input  S„((0)  and  steady-state  output  Sy(cj).  (b)  Experimental  and  theoretical  steady 
state  transfer  function  iH*(cj)i. 
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Figure  7.  (Continued.)  (c)  Experimental  and  theoretical  steady-state 
transfer  function  lH*(a;)  I . 
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L + 2 oq^ 


= eiAw(L-l)/2 
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Figure  8.  Performance  of  32-weight  ALE  with  input  of  two  sinusoids  of  unequal  power 
in  uncorrelated  noise;  delay  A=  6 samples,  = 2.83  x 10"^.  (a)  Power  spectra 

of  input  S^Cco)  and  steady-state  output  Sy(co).  (b)  Experimental  and  theoretical  steady- 
state  transfer  function  iH*(cj)i. 
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From  (30),  it  is  seen  that  IH*(cOjjy)l  obtains  a minimum  when  A satisfies  the  following 
condition; 

A + ^*  = ^-^^^^=(n+ l/2)/Af  (31) 

where  n is  any  nonnegative  integer  (such  that  (n  + l/2)/Ai'>  (L  - l)/2).  For  the  special 
case  when  Oj"  = then  a = b,  and  the  first  zero  of  IH*(cOjjy)l  occurs  when  A = (1/2) 

( 1 /AO  - (L  - 1 )/2 . For  the  case  when  Af  = 0.023  and  L = 3 2.  A 6 samples,  which  agrees 
with  the  value  of  A that  produced  the  deep  null  in  Figs.  6(b)  and  8(b).  Tlie  results  expressed 
by  (29)-(31)  and  Figs.  6(b)  and  8(b)  indicate  that  it  may  be  possible  to  improve  the  resolu- 
tion of  two  sinusoids  in  H*(a;)  by  varying  the  delay  A so  that  (3 1 ) is  satisfied.  This  variation 
of  resolution  with  A is  similar  to  the  dependence  of  the  periodogram  resolution  of  sinusoids 
on  their  initial  phase  (30,  311. 

Experimentally  measured  values  of  the  ALE  weights  and  the  derived  values  of  w*(k) 
are  compared  in  Fig.  7(b)  for  A = 19  samples.  Experimentally  measured  values  of  lH*(u))l 
for  the  ALE  and  the  theoretical  lH*(cj)l  of  the  Wiener  filter  are  compared  in  Figs.  6(b),  7(c), 
and  8(b)  and  indicate  good  agreement  between  theory  and  experiment.  As  mentioned  in 
connection  with  Fig.  4,  there  are  larger  discrepancies  between  theoretical  and  experimental 
values  of  lH*(cij)l  at  frequencies  nearer  the  cutoff  frequency  of  the  input  noise  spectrum. 

When  the  number  of  weiglits  of  the  adaptive  filter  is  large  enough  to  resolve  the  fre- 
quencies of  the  sinusoids,  the  peaks  in  lH*(u;)l  will  correspond  to  the  true  locations  of  the 
sinusoids,  as  shown  in  Fig.  9.  All  of  the  parameters  in  Fig.  9 are  in  correspondence  with  the 
parameters  in  Figs.  6-7,  except  that  A = 1 5 samples  and  L is  increased  to  128  to  provide 
improved  frequency  resolution.  The  agreement  between  theory  and  experiment  in  Fig.  9(b) 
is  seen  to  be  good.  The  experimental  curves  of  the  input  and  ALE  output  power  spectra  for 
this  case  (Fig.  9(a))  clearly  indicate  the  line  enhancement  cap'^’’'lities  of  the  ALE,  which  are 
achieved  by  suppressing  the  uncorrelated  noise. 

Table  I gives  measured  values  of  misadjustment,  as  well  as  theoretical  values  from  (7), 
corresponding  to  the  experimental  values  used  in  Figs.  6-9. 


IV.  MODIFIED  MAXIMUM  ENTROPY  SPECTRAL  ESTIMATES 
FOR  N SINUSOIDS  IN  WHITE  NOISE 


Griffiths  (41  has  defined  a useful  expression  for  purposes  of  estimating  and  tracking 
the  frequencies  of  sinusoids  in  white  noise.  This  expression  is  referred  to  in  (4)  as  a modi- 
fied maximum  entropy  spectral  estimate  and  is  defined  as  follows: 


Qx(cj)  = 


1 

II  -H*(cu)|“ 


(32) 
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Figure  9.  Perforinance  of  1 28-weight  ALE  with  input  of  two  sinusoids  of  equal  power 
in  uncorrelated  noise;  delay  A = 15  samples,  =*  3.6  x 10"^.  (a)  Power  spectra 

of  input  S ((o)  and  steady-state  output  Sy(co).  (b)  Experimental  and  theoretical  steady 
state  transfer  function  iH*(o;)i. 


Table  I.  Theoretical  and  measured  values  of  misadjustment  corresponding  to 
experiments  of  Figs.  6-9. 


Experiment 

Theoretical 

Misadjustment, 

Percent 

Measured 

Misadjustment, 

Percent 

Fig.  6 

0.01 

4 

Fig.  7 

0.01 

5 

Fig.  8 

0.01 

6 

Fig.  9 

0.05 

3 

where  H^Cco)  is  evaluated  at  A = 1 to  obtain  an  all-pole  spectral  estimate.  As  noted  in  [4) , 
differs  by  the  factor  (5)  from  the  maximum  entropy  spectral  estimate.  When 

the  input  to  the  ALE  consists  of  N sinusoids  in  white  noise,  it  is  possible  to  write  a closed 
form  expression  for  Qj^(u>)in  terms  of  the  amplitudes  A^  by  using  ( 12).  The  result  forQx(cj)is 


Qx(w)  = 


2N 

I J 

n=l 


1 - e 

, j(w  -w) 

1 - e " 


(33) 


where  the  A„  are  evaluated  with  A = 1 . It  is  interesting  to  note  that  evaluated  at  the 

frequency  of  the  rth  sinusoid  is  only  a function  of  A,,  as  well  as  the  SNR  of  the  rth^  sinusoid. 
This  can  be  seen  with  the  use  of  (13)  and  (33).  The  result  for 


Qx(cjr)  = lArf^  {a^'^/loQh^.  (34) 

Eq.  (34)  is  valid  regardless  of  the  frequency  separation  between  the  N sinusoids  (]^.,  regard- 
less of  the  values  of  the  7^)-  (3f  course,  when  the  frequency  separations  are  very  small,  the 

values  Qx(<*’r^  necessarily  correspond  to  the  maxima  of  Qx(w).  Note  that  as  all  the 

coupling  coefficients  approach  0,  the  A^  approach  exp  (jw^)/(L  + and  Qx(9Jr^ 

is  given  approximately  by 

Qx(Wj)  a (a//4ao'*)  (L  + la^p-lo^h^  ■ (35) 


When  L » 2/SNRj.  (SNR^  = o^Ioq),  Qx(w^)  will  be  given  by 


(36) 
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Eq.  (34)  shows  that  the  value  Qjj(cOj.)  is,  in  effect,  influenced  by  all  of  the  other  sinusoids  at 
^ ~ through  coupling  terms  Eqs.  (35)-(36)  show  that,  as  these  terms 

approach  zero,  Qj^fcUj.)  becomes  independent  of  all  the  other  sinusoids  and  depends  only  on 

the  SNR  of  the  sinusoid.  Similar  results  have  also  been  obtained  by  Lacoss  (12}  for  the 
theoretical  peak  values  of  the  maximum  entropy  estimate  of  one  sinusoid  in  white  noise  using 
Woodbury’s  identity.  A further  discussion  of  these  theoretical  results  as  well  as  similar  results 
corresponding  to  sinusoids  in  one-pole  low-pass  noise  is  given  in  [32] . 

For  the  purpose  of  comparing  and  H*{g;),  plots  of  the  modified  maximum 

entropy  spectral  estimate  defined  by  (32)  are  presented  in  Fig.  10  corresponding  to 

the  frequencies,  SNR’s,  and  filter  length  of  Figs.  6-8.  The  plots  of  *0  were 

calculated  from  the  theoretical  values  of  H*(co)  computed  from  the  Fourier  transform  of 

the  w*(k)  given  by  (12).^  It  is  seen  in  Figs.  10(a)  and  10(b)  that,  in  contrast  to  the  plots  of 
iH^fco)!  in  Figs.  6(b),  7(c),  and  8(b),  the  theoretical  estimate  gives  the  frequency 

location  of  the  sinusoids  more  accurately. 

^It  should  be  noted  that  Griffiths  (41  has  shown  that  the  instantaneous  adaptive  esti- 
mate Qx(w,  j),  which  is  defined  as  in  (32),  with  H*(w)  replaced  by 

L-l 

^ Wj(k)  ^ 

k=0 

where  Wj(k)  is  obtained  from  (1 ) with  A = 1 , provides  a good  frequency  estimate  of  sinusoids 
in  white  noise,  especially  for  high  input  SNR  cases.  The  theoretical  plots  of  Q^fw)  in  Fig.  10 
indicate  that  even  when  the  input  SNR’s  are  not  very  large,  Qj^(a);  j)  may  still  provide  a very 

good  frequency  estimate  of  the  input  signals.  However,  as  the  input  noise  is  increased,  the 
^riance  in  the  ALE  weight  vector  noise  increases  (for  p fixed),  and  the  confidence  in 

3 spectral  estimate  will  be  degraded  to  some  extent.  Baggeroer  (27] , in  deriving 

confidence  intervals  for  maximum  entropy  (MEM)  spectral  estimates,  has  considered  the 
relevant  case  of  several  closely  spaced  sinusoids  in  white  noise.  For  this  case,  Baggeroer  has 
shown  that  although  the  MEM  has  very  large  peak-to-background  ratios,  the  confidence  inter- 
vals associated  with  these  peaks  can  be  quite  large.  In  deriving  the  MEM  confidence  intervals, 
Baggeroer  assumed  that  the  MEM  estimate  is  formed  by  segmenting  the  input  data  sequence 
into  constant  length  data  blocks.  Although  this  particular  method  of  segmenting  the  data 
into  blocks  is  used  in  most  direct,  or  FFT,  methods  of  spe^ral  estimation,  it  is  not  the  method 
which  is  used  to  form  the  adaptive  MEM  spectral  estimate  j).  In  contrast  to  segmenting 

the  input  data,  the  ALE  (as  noted  in  ( 10] ) works  on  a steady  flow  of  data  that  it  weights 
exponentially  over  time.  Therefore,  unresolved  questions  arise  with  respect  to  the  confidence 
(especially  at  the  peaks)  of  the  adaptive  MEM  spectral  estimate.  Of  course,  the  confidence 


^Corresponding  experimental  plots  of  Qx(w)  were  not  available  from  the  particular  hardware  implementa- 
tion of  the  ALE  used  in  the  experiments. 
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Figure  10.  Theoretical  performance  of  32-weighi  ALE  implemented  as  a maximum  en- 
tropy frequency  tracker  with  delay  A=l.  (a)  With  input  of  two  sinusoids  of  equal  power 
in  uncorrelated  noise,  (b)  With  input  of  two  sinusoids  of  unequal  power  in  uncorrelated 
noise. 
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intervals  for  Qx(w;j)  can  be  reduced  by  decreasing  /i;  however,  this  will  also  increase  the 
ALE  data  processing  time.  Therefore,  the  usefulness  of  Purposes  of  real-time 

spectral  estimation  in  noisy  environments  is  not  yet  understood. 


V.  CONCLUSIONS 


The  steady-state  behavior  of  the  ALE  has  been  analyzed  in  this  paper  for  stationary 
inputs  consisting  of  multiple  sinusoids  in  white  noise  using  constrained  Wiener  filter  theory. 

An  analytic  solution  for  an  L-weight  discrete  Wiener  filter  has  been  derived  for  N sinusoids 
in  white  noise.  This  solution  shows  that  the  expected  values  of  the  ALE  weights  in  steady 
state  can  be  written  as  a sum  of  sinusoids  and  that  the  amplitude  of  each  sinusoid  A^ 

(n  = 1 , 2, . . . , 2N)  is  coupled  to  the  amplitude  of  all  the  other  sinusoids  by  coefficients  which 
approach  zero  as  the  ALE  filter  length  becomes  large.  This  particular  representation  for  the 
steady-state  weights  is  equivalent  to  a transformation  of  the  L X L Wiener-Hopf  matrix 
equation  to  a set  of  2N  coupled  linear  equations  for  the  amplitudes  Aj^.  When  N>  2,  the 

transformation  method  becomes  much  simpler  to  use  than  Woodbury’s  identity  (which  has 
been  used  considerably  by  a number  of  authors  to  examine  maximum  entropy  filtering  of 
sinusoids  in  white  noise;  see  e.g.  [12],  [131  and  [30]).  Comparisons  between  the  theoretical 
expressions  derived  in  Section  II  and  experimentally  measured  impulse  responses  and  trans- 
fer functions  of  the  steady-state  ALE  (obtained  from  an  ALE  hardware  implementation)  for 
one  and  two  sinusoidal  inputs  in  white  noise  were  presented  in  Section  III  and  indicate  the 
validity  of  the  analytical  models  used  in  Section  11.  The  modified  maximum  entropy  spectral 
estimate  was  also  analyzed  in  Section  IV  for  multiple  sinusoidal  inputs  in  white  noise. 

Since  the  analytical  results  presented  in  this  paper  were  obtained  on  the  basis  of  constrained 
Wiener  filter  theory  for  a finite  length  linear  prediction  filter  with  arbitrary  prediction  dis- 
tance, they  describe  the  performance  of  a much  wider  class  of  linear  predictive  filters  than  the 
specific  implementation  known  as  the  ALE. 

Extensions  of  the  analytical  results  given  in  this  paper  to  more  general  types  of 
rational  input  spectra  are  currently  being  carried  out  and  will  be  presented  in  a future  paper. 

It  has  been  shown  that  the  method  of  undetermined  coefficients  also  provides  useful  analyti- 
cal solutions  of  the  discrete  Wiener-Hopf  matrix  equation  (3)  for  general  rational  input 
spectra  [28). 


APPENDIX 

The  purpose  of  this  appendix  is  twofold.  First,  it  will  be  shown  that  g is  positive 
when  0xx^*^^  given  by  (1 1)  with  Oq^  + 0.  Second,  it  will  be  shown  that  (13)  has  a unique 

^ ‘ 

solution  when  oqp  + 0.  The  proof  of  the  first  statement  is  trivial  and  consists  of  examining  ■ 

J 
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r 


1 


the  quadratic  form  of  R.  Let  x be  a nonzero  L-dimensional  column  vector  and  denote  by  jc* 
the  transpose  of  Then  the  quadratic  form  of  R using  the  notation  of  (1 2)-(13),  is  given  as 


L-1 

2N 

L 

-1  L-1 

|Xr|2+  1/2  y 

r=0 

k=l 

r=0  n=0 

L-1 

2N 

L-1 

= V 2 

\x/+i/2  y 

2 ’‘r 

r=0 

k=l 

-jWkr 


(A.1) 


However,  the  last  expression  for  i ^ strictly  positive  for  nonzero  x,  and  there- 

fore R is  positive  definite. 

To  see  that  (13)  has  a unique  solution  when  Oq^  f 0,  note  that  it  may  be  expressed 
in  the  equivalent  form  (by  multiplying  the  rth  equation  of  (13)  by  L + 


B-  A = C 

where  B is  a 2N  X 2N  Hermitian  matrix  with  elements 


(A.  2) 


(B^)rn  = 6(r  - n)  + ^ 

k=0 


A is  a 2N-dimensional  column  vector  with  components  (A)^  = A,^;  and  C is  a 2N-dimensional 
column  vector  with  components  (C)j.  = exp  (jcOjA).  To  show  that  (A.2)  has  a unique  solution 
we  will  show  that  B is  positive  definite.  Let  x be  a nonzero  2N-dimensional  column  vector 
and^^  be  its  complex  conjugate  transpose.  Then,  the  quadratic  form  of  B is  written  as 

2N  2N  2N  L-1 

% |Xr|2  (2aoX^)  + ^ ^ x„ 

r=l  r=l  n=l  lc=0 


2N  L-1 

= j 1x^12(200^/0^2)+ ^ 

1=1  k=0 


2N 

Iv 

r=l 


jwrk 


(A.3) 


As  observed  in  connection  with  (A.l),  the  last  expression  for^^  2 (A.3)  is  strictly 

positive  for  nonzero  ^ and  therefore  B is  positive  definite. 
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